MGHRL: Meta Goal-generation for Hierarchical Reinforcement Learning

Most meta reinforcement learning (meta-RL) methods learn to adapt to new tasks by directly optimizing the parameters of policies over primitive action space. Such algorithms work well in tasks with relatively slight difference. However, when the task distribution becomes wider, it would be quite inefficient to directly learn such a meta-policy. In this paper, we propose a new meta-RL algorithm called Meta Goal-generation for Hierarchical RL (MGHRL). Instead of directly generating policies over primitive action space for new tasks, MGHRL learns to generate high-level meta strategies over subgoals given past experience and leaves the rest of how to achieve subgoals as independent RL subtasks. Our empirical results on several challenging simulated robotics environments show that our method enables more efficient and generalized meta-learning from past experience.Comment: Accepted to the ICLR 2020 workshop: Beyond tabula rasa in RL (BeTR-RL

Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learning

Context, the embedding of previous collected trajectories, is a powerful construct for Meta-Reinforcement Learning (Meta-RL) algorithms. By conditioning on an effective context, Meta-RL policies can easily generalize to new tasks within a few adaptation steps. We argue that improving the quality of context involves answering two questions: 1. How to train a compact and sufficient encoder that can embed the task-specific information contained in prior trajectories? 2. How to collect informative trajectories of which the corresponding context reflects the specification of tasks? To this end, we propose a novel Meta-RL framework called CCM (Contrastive learning augmented Context-based Meta-RL). We first focus on the contrastive nature behind different tasks and leverage it to train a compact and sufficient context encoder. Further, we train a separate exploration policy and theoretically derive a new information-gain-based objective which aims to collect informative trajectories in a few steps. Empirically, we evaluate our approaches on common benchmarks as well as several complex sparse-reward environments. The experimental results show that CCM outperforms state-of-the-art algorithms by addressing previously mentioned problems respectively.Comment: Accepted to AAAI 202

Temperature- and field angular-dependent helical spin period characterized by magnetic dynamics in a chiral helimagnet MnNb3S6MnNb_3S_6

The chiral magnets with topological spin textures provide a rare platform to explore topology and magnetism for potential application implementation. Here, we study the magnetic dynamics of several spin configurations on the monoaxial chiral magnetic crystal $MnNb_3S_6$ via broadband ferromagnetic resonance (FMR) technique at cryogenic temperature. In the high-field forced ferromagnetic state (FFM) regime, the obtained frequency f vs. resonance field Hres dispersion curve follows the well-known Kittel formula for a single FFM, while in the low-field chiral magnetic soliton lattice (CSL) regime, the dependence of Hres on magnetic field angle can be well-described by our modified Kittel formula including the mixture of a helical spin segment and the FFM phase. Furthermore, compared to the sophisticated Lorentz micrograph technique, the observed magnetic dynamics corresponding to different spin configurations allow us to obtain temperature- and field-dependent proportion of helical spin texture and helical spin period ratio L(H)/L(0) via our modified Kittel formula. Our results demonstrated that field- and temperature-dependent nontrivial magnetic structures and corresponding distinct spin dynamics in chiral magnets can be an alternative and efficient approach to uncovering and controlling nontrivial topological magnetic dynamics.Comment: 29 pages (including Supporting Information), 7 figures, accepted by SCIENCE CHINA Physics, Mechanics & Astronom

ImMesh: An Immediate LiDAR Localization and Meshing Framework

In this paper, we propose a novel LiDAR(-inertial) odometry and mapping framework to achieve the goal of simultaneous localization and meshing in real-time. This proposed framework termed ImMesh comprises four tightly-coupled modules: receiver, localization, meshing, and broadcaster. The localization module utilizes the prepossessed sensor data from the receiver, estimates the sensor pose online by registering LiDAR scans to maps, and dynamically grows the map. Then, our meshing module takes the registered LiDAR scan for incrementally reconstructing the triangle mesh on the fly. Finally, the real-time odometry, map, and mesh are published via our broadcaster. The key contribution of this work is the meshing module, which represents a scene by an efficient hierarchical voxels structure, performs fast finding of voxels observed by new scans, and reconstructs triangle facets in each voxel in an incremental manner. This voxel-wise meshing operation is delicately designed for the purpose of efficiency; it first performs a dimension reduction by projecting 3D points to a 2D local plane contained in the voxel, and then executes the meshing operation with pull, commit and push steps for incremental reconstruction of triangle facets. To the best of our knowledge, this is the first work in literature that can reconstruct online the triangle mesh of large-scale scenes, just relying on a standard CPU without GPU acceleration. To share our findings and make contributions to the community, we make our code publicly available on our GitHub: https://github.com/hku-mars/ImMesh

A Study of Pulsation properties of 57 Non-Blazhko effect ab-type RR Lyrae stars with homogeneous metallicities from the LAMOST-Kepler/K2 survey

Homogeneous metallicities and continuous high-precision light curves play key roles in studying the pulsation properties of RR Lyrae stars. By cross-matching with LAMOST DR6, we have determined 7 and 50 Non-Blazhko RRab stars in the Kepler and K2 fields, respectively, who have homogeneous metallicities determined from low-resolution spectra of the LAMOST-Kepler/K2 project. The Fourier Decomposition method is applied to the light curves of these stars provided by the Kepler space based telescope to determine the fundamental pulsation periods and the pulsation parameters. The calculated amplitude ratios of R21, R31 and the phase differences of {\phi}21, {\phi}31 are consistent with the parameters of the RRab stars in both the Globular Clusters and the Large Magellanic Cloud. We find a linear relationship between the phase differences {\phi}21 and {\phi}31, which is in good agreement with the results in previous literature. As far as the amplitude, we find that the amplitude of primary frequency A1 and the total amplitude Atot follow either a cubic or linear relationship. For the rise time RT, we do not find its relevance with the period of the fundamental pulsation mode P1, or Atot and {\phi}21. However, it might follow a linear relationship with R31. Based on the homogeneous metallicities, we have derived a new calibration formula for the relationship of period-{\phi}31-[Fe/H], which agrees well with the previous studies

Joint Intrinsic and Extrinsic LiDAR-Camera Calibration in Targetless Environments Using Plane-Constrained Bundle Adjustment

This paper introduces a novel targetless method for joint intrinsic and extrinsic calibration of LiDAR-camera systems using plane-constrained bundle adjustment (BA). Our method leverages LiDAR point cloud measurements from planes in the scene, alongside visual points derived from those planes. The core novelty of our method lies in the integration of visual BA with the registration between visual points and LiDAR point cloud planes, which is formulated as a unified optimization problem. This formulation achieves concurrent intrinsic and extrinsic calibration, while also imparting depth constraints to the visual points to enhance the accuracy of intrinsic calibration. Experiments are conducted on both public data sequences and self-collected dataset. The results showcase that our approach not only surpasses other state-of-the-art (SOTA) methods but also maintains remarkable calibration accuracy even within challenging environments. For the benefits of the robotics community, we have open sourced our codes

Swashplateless-elevon Actuation for a Dual-rotor Tail-sitter VTOL UAV

In this paper, we propose a novel swashplateless-elevon actuation (SEA) for dual-rotor tail-sitter vertical takeoff and landing (VTOL) unmanned aerial vehicles (UAVs). In contrast to the conventional elevon actuation (CEA) which controls both pitch and yaw using elevons, the SEA adopts swashplateless mechanisms to generate an extra moment through motor speed modulation to control pitch and uses elevons solely for controlling yaw, without requiring additional actuators. This decoupled control strategy mitigates the saturation of elevons' deflection needed for large pitch and yaw control actions, thus improving the UAV's control performance on trajectory tracking and disturbance rejection performance in the presence of large external disturbances. Furthermore, the SEA overcomes the actuation degradation issues experienced by the CEA when the UAV is in close proximity to the ground, leading to a smoother and more stable take-off process. We validate and compare the performances of the SEA and the CEA in various real-world flight conditions, including take-off, trajectory tracking, and hover flight and position steps under external disturbance. Experimental results demonstrate that the SEA has better performances than the CEA. Moreover, we verify the SEA's feasibility in the attitude transition process and fixed-wing-mode flight of the VTOL UAV. The results indicate that the SEA can accurately control pitch in the presence of high-speed incoming airflow and maintain a stable attitude during fixed-wing mode flight. Video of all experiments can be found in youtube.com/watch?v=Sx9Rk4Zf7sQComment: 8 pages, 13 figure

Ammonium fluoride additive-modified interphase chemistry stabilizes zinc anodes in aqueous electrolytes

Herein, ammonium fluoride is reported as an additive within 1 M ZnSO4 aqueous electrolyte to improve zinc anodes. The as-formed electrostatic shielding layer and ZnF2-rich solid-state interphase layer can jointly inhibit side reactions and dendrite growth. Consequently, symmetric Zn‖Zn cells, asymmetric Zn‖Cu cells and Zn‖MnO2 cells with the additives present dramatically enhanced performance in comparison to the ones with pure ZnSO4 electrolyte counterparts. This work proposes a facile but effective method to achieve highly reversible zinc anodes